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Description 

System and Method for Monitoring Visible Changes 

5 This application is a Continuation-in-Part of utility application Serial No. 

09/258,056. 

Background of the Invention 

1 . Field of the Invention 

1 0 The present invention relates generally to methods and apparatus for video or 

other photographic security systems. 

2. Description of Related Art 

Security systems using video cameras have been in use for many years. 
1 5 United States patent number 5,099,322 shows a system for detecting scene changes 
using a video security system. United States patent number 5,471, 239 show the 
system for using video frame compression and numerical comparison to detect scene 
changes. 

20 Summary of the Invention 

In a method for monitoring a field of view for visible changes, according to 
the present invention, a first or benchmark image is taken in a predetermined manner 
of the field of view to be monitored, and then an array divides the first or benchmark 
image into a plurality of cells. The first or benchmark image divided by the array is 

2 5 then stored, and a second image is taken in the predetermined manner of the field of 
view to be monitored. The array is also applied to the second image, dividing the 
second image into the plurality of cells. Cells are combined into predetermined 
groups. The groups could be individual cells, which is to say groups of one cell 
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each. The groups could be rows of cells or columns of cells or rectangular 
groupings which in effect create larger cells. The predetermined groups of cells of 
the second image are then compared to the same groups of cells of the first or 
benchmark image, and the number of the groups of cells that changed from the first 
5 or benchmark image to the second image is computed. In some situations only 

certain groups of cells are compared. If a parrot is moving around in a cage, then the 
cells or the groups of cells which might show the parrot are not compared, or if there 
are several ceiling fans which are moving, then the cells or the groups of cells which 
show the ceiling fans might not be compared. There are also other ways of handling 

1 0 these problems which are also described. 

In one preferred form of the method, each cell of the first or benchmark 
image is given a numerical value based upon the information in each cell, and each 
cell of the second image is also given a numerical value based upon the information 
in each cell. Each group of cells is also given a numerical value based on the 

1 5 numerical value of the cells within the group. The simplest method is to make the 
numerical value of each group equal to the sum of the numerical values of the cells 
within the group, but manipulation would work as long as the numerical value of the 
group changes when the numerical value of its cells change. Computing how many 
groups of the cells changed from the first or benchmark image to the second image is 

2 0 simply a matter of computing the difference between the numerical value of each 
group of cells in the second image and in the first or benchmark image. 

In one arrangement, each cell is a single picture element, known as a pixel. A 
pixel has a its own brightness ranging from 0 for black to a maximum value, 
typically 255, for white, and the numerical value of each cell is the brightness of the 

2 5 pixel. There is still a brightness number for color images, but color scales could also 
be used. The numerical value of each group of cells is determined by an image 
algorithm, which in one preferred form is simply taking the sum of the brightness for 



the pixels in the group. In one such arrangement, the predetermined groups of cells 
comprises a single group. 

In one method according to the present invention, the number of the groups 
of cells which changed in numerical value by more than a predetermined amount is 
also computed. 

One method, according to the present invention, also includes the step of 
comparing the number of the groups of cells that changed from the first or 
benchmark image to the second image to a predetermined number. If the parrot 
moving in its cage, or a cat climbing on the furniture is small enough to never be in 
more than the predetermined number groups of cells, then if more than the 
predetermined number groups of cells changed, then it was not caused by the parrot 
or the cat. 

Another method, according to the present invention, for monitoring a field of 
view for visible changes, includes the steps of taking a sequence of video images in 
a predetermined manner of the field of view to be monitored, creating a sequence of 
images, and creating an array which divides each of the images into a plurality of 
cells. Each image divided by the array is stored and the groups of cells of each 
image are compared to the same groups of cells of the other images. Again, this 
could just be predetermined groups of cells which are less than all of the groups of 
cells. It is then computed if there is a cyclical change in any of the predetermined 
groups of cells during the sequence. This cyclical change could be the rotation of a 
ceiling fan or the movement of a flower arrangement caused by the blowing of an air 
conditioner. Similar to that already described, each cell of each image is given a 
numerical value based upon the information in that cell, and each group of cells is 
given a numerical value based upon the numerical value of the cells within that 
group, and computing if there is a cyclical change in any of the predetermined 
groups of cells during the sequence becomes computing if there is a cyclical change 
in any of the numerical values of the predetermined groups of cells during the 



sequence. In a preferred form, the number of the predetermined groups of cells that 
changed from one image of the sequence to the next image of the sequence and was 
not part of a cyclical change is computed. 

These and other objects, advantages and features of this invention will be 
apparent from the following description taken with reference to the accompanying 
drawing, wherein is shown a preferred embodiment of the invention. 

Brief Description of the Drawing 

FIGURE 1 is a block diagram representation of a system for monitoring a 
field of view for visible changes in accordance with a present invention; 

FIGURE 2 is a flow diagram of a method for monitoring a field of view for 
visible changes in accordance with the present invention; 

FIGURE 3 is a representation of a first or benchmark image and array in 
accordance with the present invention; 

FIGURE 4 is a representation of a second image and array in accordance 
with a present invention; 

FIGURE 5 is an alternative representation of a second image and array in 
accordance with present invention; and 

FIGURE 6 is a representation of the first or benchmark image and array of 
Figure 3 showing the use of a preferred algorithm. 

Description of the Preferred Embodiment 

Referring now to the drawing, and in particular to Figure 1 , a system 
according to the present invention is referred to generally by reference to 10. 
Surveillance camera 12 can be in a fixed position and angle with respect to field of 
view 14 to be monitored, although this is not necessary, and can even work with a 
fixed lens so that any image that it takes of field of view 14 will be identical except 
for changes in field of view 14 itself. Images from video camera 12 are fed to 



computer 16 where they are input as video images via video image grabber 18. 
Video image grabber 18 can be anything compatible with the particular video 
camera, such as a 256 gray scale or a red, green, blue (RGB) scale adapter. The 
video images are transmitted from video image grabber 1 8 to both video recorder 20 
and image processor 22. Once each image has been processed, the information 
about that image is made available to the surveillance application program 24. 
Video recorder 20 can be a video cassette recorder (VCR) or a digital versatile disk 
(DVD, formerly "digital video disk") or some other kind of recorder. 

Referring also to Figure 2, Figure 3, Figure 4 and Figure 5, method for 
monitoring a field of view for visible changes, according to the present invention, is 
referred to generally by reference in 26. After the video image grabber 18 is 
initialized 28, a first or benchmark or benchmark image 30 is taken by video camera 
12 in a predetermined manner of the field of view 14 to be monitored, which is read 
32, and then an array 34, which can be a matrix or grid or other array, divides the 
first or benchmark image into a plurality of cells 36. The first or benchmark image 
divided by the array is then stored, and a second image 38 is taken in the 
predetermined manner of the field of view to be monitored. Array 34 is also applied 
to the second image, dividing the second image into the plurality of cells. The cells 
of each row R2-1 through R2-10 of the second image are grouped and the cells of 
each row Rl-1 through Rl-10 of the first or benchmark image are grouped 42. The 
rows of the second image are then compared to the rows of the first or benchmark 
image, and the number of the rows that changed from the first or benchmark image 
to the second image is computed. This could, of course, be done using columns 
rather than rows, and in a preferred form, rows and columns are alternately 
compared. In some situations only certain groups are compared. If a parrot is 
moving around in a cage, then the cells which might show the parrot are not 
compared, or if there are several ceiling fans which are moving, then the cells which 
show the ceiling fans might not be compared by the comparison algorithm. There 



are also other ways of handling these problems which are also shown. In the 
situation illustrated in Figure 4, a cat 46 has moved into field of view 14 explained 
the first or benchmark image and the second image. Cat 46 can occupy up to three 
rows. The compare against standard 44 portion of the program can simply be 
instructed to ignore changes in no more than three rows so that the cat will not give a 
false alarm. Referring now to figure 5, an alternative second image is referred to by 
reference number 48. In this case, a human being 50 has entered field of view 14. 
Human being 50 occupies all ten rows so that there is no danger in the present 
invention of thus taking human being 50 for cat 46 for vice versa. If columns were 
used in this case, the man could take two to three columns as could the cat. 
Alternating comparisons of rows and columns, or comparing both, avoids mistakes 
from such situations. 

Figure 5 could, of course, be the first or benchmark image and Figure 3 be 
the second image, in which case, the present system would detect human being 50 
leaving field of vision 14. In such a case, since human being 50 occupies the same 
space as he moves, he can move within the image without triggering an alarm, 
depending on the comparison algorithm used. 

In one preferred form of the method, each cell of the first or benchmark 
image is given a numerical value based upon the information in each cell, and each 
cell of the second image is also given a numerical value based upon the information 
in each cell. Computing how many groups of the cells changed from the first or 
benchmark image to the second image is simply a matter of computing the 
difference between the numerical value of each cell in the second image and in the 
first or benchmark image. 

In a preferred arrangement, each cell comprises a pixel, the numerical value 
of each cell comprises the brightness of the pixel; and the numerical value of each 
group of cells comprises the sum of the brightness for the pixels in the group. In an 
arrangement wherein the predetermined groups of cells comprises a single group, the 



comparison number is a single number that can be quickly calculated since it is 
simply the sum of the pixel brightness numbers. Then the single number is 
compared for the first or benchmark image and the second image. This can be 
implemented with a minimum of computer power, making it easy to monitor many 
different cameras. In another arrangement, each group of cells is a row of cells, 
wherein a single number can be obtained by a sum of row sums. It would work the 
same for columns. Once again, a sum of row sums could be used alternately with a 
row of column sums or both sum of row sums and column sums, again obtaining a 
single number for comparison, would work as well. 

In a more general sense, if we define "image mass" as anything which can be 
measured about an image, such as the sum of the brightness numbers for all of the 
pixels, then a preferred form of the present invention is to compare the image mass 
of the first or benchmark image with the image mass of the second image. Such a 
system makes it possible for a guard station to operate with a single monitor, 
television or other video screen. Whenever a significant change is detected, then the 
monitor, television or other video screen is switched to that camera where the 
change was detected, while sounding an audible alarm at the guard station to alert 
the guard that a change has been detected. Besides relieving the tedium of 
monitoring many different sites where nothing is happening, it would simplify 
monitoring so that many different sites could actually be monitored at a police 
station. Once the alarm has been sounded, then whoever is monitoring the site can 
display both the benchmark image and the image which changed enough to trigger 
the alarm in addition to the current image. A video recording begins to record the 
live action of the surveillance camera 12 on video recorder 20, or can view a live 
image directly from the surveillance camera. If it appears to be a false alarm, then 
the person monitoring the site can reset a new benchmark from the computer. 



In one method according to the present invention, the number of the cells or 
groups of cells and which changed in numerical value by more than a predetermined 
amount is also computed. 

One method, according to the present invention, also includes the step of 
comparing 44 the number of the groups of cells that changed from the first or 
benchmark image to the second image to a predetermined number. If the parrot 
moving in its cage, or a cat climbing on the furniture is small enough to never be in 
more than the predetermined number of groups of cells, then if more than the 
predetermined number of groups of cells changed, then it was not caused by the 
parrot or the cat. 

Another method, according to the present invention, for monitoring a field of 
view for visible changes, includes the steps of taking a sequence of video images in 
a predetermined manner of the field of view to be monitored, creating a sequence of 
images, and creating an array which divides each of the images into a plurality of 
cells. Each image divided by the array is stored and groups of the cells of each 
image are compared to the same groups of cells of the other images. Again, this 
could just be predetermined groups of cells which add to less than all of the cells. It 
is then computed if there is a cyclical change in any of the predetermined groups of 
cells during the sequence. This cyclical change could be the rotation of a ceiling fan 
or the movement of a flower arrangement caused by the blowing of an air 
conditioner. Similar to that already described, each cell of each image is given a 
numerical value based upon the information in that cell, and computing if there is a 
cyclical change in any of the predetermined groups of cells during the sequence 
becomes computing if there is a cyclical change in any of the numerical values of the 
predetermined groups of cells during the sequence. In a preferred form, the number 
of the predetermined cells that changed from one image of the sequence to the next 
image of the sequence and was not part of a cyclical change is computed. 



Referring now specifically to Figure 6, one image algorithm according to the 
present invention sums rows or columns of cells by assigning each cell a unique 
numerical value. In the case illustrated, the cells are numbered from left to right 
beginning at the top. The value of each cell is added to the value of a row or column 
if a visible item occupies part of that cell. For the field of view illustrated, a table 
occupies cells 41, 42,43,51,52,53,61,62,63,71 and 72. A bookcase occupies cells 4, 
5, 6, 14, 15, 16, 24, 25, 26, 34, 35, 36, 44, 45, 46, 54, 55, 56, 64, 65 and 66. A chair 
occupies cells 39, 48,49, 50, 58,59,60, 68,69,70,78,79 and 80. The value of groups, 
where a group is a row, far from the top: 15, 45, 75, 144, 408, 498, 588, 380, zero 
and zero. This algorithm has several advantages over using image compression. 
The algorithm is very simple, for each cell has affixed value which uses that value 
when the cell is occupied and a value of zero when that cell is not occupied. The 
value of a row is equal to the sum of the occupied cell numbers in that row. 
Similarly, the value of a column would be the sum of the occupied cell numbers in 
that column. The sum of row sums or the sum of column sums is the number 2153 
shown in the lower right corner. As before, sum of row sums could be alternated 
with row of column sums or the two could be added together, in which case the 
number in the lower right corner would be doubled. 

From the foregoing it will be seen that this invention is well adapted to 
attain all of the ends and objectives hereinabove set forth, together with other 
advantages which are inherent to the apparatus. 

It will be understood that certain features and subcombinations are of utility 
and may be employed without reference to other features and subcombinations. This 
is contemplated by and is within the scope of the claims. 

As many possible embodiments may be made of the invention without 
departing from the scope thereof, it is to be understood that all matter herein set 
forth or shown in the figures of the accompanying drawings is to be interpreted as 
illustrative and not in a limiting sense. 



